Systems, devices, and methods for generating a social street view

ABSTRACT

Disclosed are systems, devices, and methods for creating a rendering of real-world locations with embedded multimedia elements. An exemplary method includes receiving image data of a real-world location, identifying geographic coordinates of the real-world location and/or a point of view from which the image data was acquired, acquiring multimedia elements relevant to the real-world location based on the geographic coordinates and/or the point of view, and creating a rendering of the image data with the multimedia elements embedded therein.

PRIORITY CLAIM

The present application is a 371 application which claims priority toand the benefit of PCT Application No. PCT/US2016/023429, filed on Mar.21, 2016, which claims priority to and the benefit of U.S. ProvisionalPatent Application No. 62/135,941 by Ruofei Du and Amitabh Varshney,entitled “SYSTEM AND METHOD FOR SOCIAL STREET VIEW,” filed on Mar. 20,2015, the entire contents of which are incorporated herein by reference.

GOVERNMENT SUPPORT

This invention was made with government support under CNS1429404 awardedby the National Science Foundation. The government has certain rights inthe invention.

BACKGROUND Technical Field

The present disclosure relates to the generation of a social streetview, and, more particularly, to systems, devices, and methods forassembling, organizing, and formatting multimedia elements from varioussources, and generating renderings of real-world locations incorporatingthe multimedia elements.

Description of Related Art

Social media have become a ubiquitous part of how people shareinformation with each other. Millions of people share and/or view socialmedia content, such as pictures, videos, textual, and audio messagesevery day. However, most social media platforms generally only allowsuch social media content to be viewed in a linear manner, i.e.,pictures may only be viewed one at a time and in sequence. Some socialmedia platforms may organize the social media content in atwo-dimensional layout, but the organization is often arbitrary. Thereexists a need for improvement in the manner in which social mediacontent is displayed to and experienced by a viewer.

SUMMARY

Provided in accordance with the present disclosure is a method forcreating a rendering of real-world locations with embedded multimediaelements. In an aspect of the present disclosure, the method includesreceiving image data of a real-world location, identifying geographiccoordinates of the real-world location and/or a point of view from whichthe image data was acquired, acquiring multimedia elements relevant tothe real-world location based on the geographic coordinates and/or thepoint of view, and creating a rendering of the image data with themultimedia elements embedded therein.

In another aspect of the present disclosure, the method further includesdisplaying the rendering of the image data with the multimedia elementsembedded therein.

In a further aspect of the present disclosure, the multimedia elementsare geo-tagged, and wherein the multimedia elements are acquired basedon geo-tag information of the multimedia elements.

In another aspect of the present disclosure, the multimedia elementsinclude pictures.

In a further aspect of the present disclosure, the multimedia elementsinclude videos.

In another aspect of the present disclosure, the multimedia elementsinclude textual messages, gestures, graffiti, or audio messages.

In a further aspect of the present disclosure, the multimedia elementsinclude sentiments derived from pictures, videos, textual messages,gestures, graffiti, or audio messages.

In another aspect of the present disclosure, the multimedia elements areembedded in the image data in a manner which fuses the multimediaelements with the image data.

In a further aspect of the present disclosure, fusing the multimediaelements with the image data includes determining a geometry of thereal-world location based on the image data, and aligning the multimediaelements with the determined geometry of the real-world location.

In yet a further aspect of the present disclosure, aligning themultimedia elements with the determined geometry of the real-worldlocation includes displaying the multimedia elements on the sides ofbuildings in the image data.

In another aspect of the present disclosure, aligning the multimediaelements with the determined geometry of the real-world locationincludes displaying the multimedia elements on virtual billboards in theimage data.

In yet another aspect of the present disclosure, the image data includestill images of the real-world location.

In a further aspect of the present disclosure, the image data includevideo images of the real-world location.

In another aspect of the present disclosure, displaying the rendering ofthe image data with the multimedia elements embedded therein includesdisplaying the rendering using a virtual reality or augmented realitydisplay.

In a further aspect of the present disclosure, the multimedia elementsare received from social media platforms.

In another aspect of the present disclosure, the method further includesfiltering the acquired multimedia elements based on user-definedparameters.

In a further aspect of the present disclosure, the user-definedparameters include a date, a time, a season, a person, group of people,a social media profile, a keyword, a distance from the real-worldlocation, or a data type of the multimedia elements.

Provided in accordance with the present disclosure is a system forcreating a rendering of real-world locations with embedded multimediaelements. In an aspect of the present disclosure, the system includes acomputing device including a processor and a memory storing instructionswhich, when executed by at least one processor, causes the computingdevice to, receive image data of a real-world location, identifygeographic coordinates of the real-world location and/or a point of viewfrom which the image data was acquired, acquire multimedia elementsrelevant to the real-world location based on the geographic coordinatesand/or the point of view, create a rendering of the image data with themultimedia elements embedded therein, and send the rendering of theimage data with the multimedia elements embedded therein, and a displaydevice configured to, receive the rendering of the image data with themultimedia elements embedded therein, and display the rendering of theimage data with the multimedia elements embedded therein.

In another aspect of the present disclosure, the display device is avirtual reality or augmented reality display device.

Provided in accordance with the present disclosure is a non-transitorycomputer-readable storage medium storing instructions which, whenexecuted by at least one processor, cause a computer to receive imagedata of a real-world location, identify geographic coordinates of thereal-world location and a point of view from which the image data wasacquired, acquire multimedia elements relevant to the real-worldlocation based on the geographic coordinates and the point of view,create a rendering of the image data with the multimedia elementsembedded therein, and display the rendering of the image data with themultimedia elements embedded therein.

Any of the above aspects and embodiments of the present disclosure maybe combined without departing from the scope of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other advantages will become more apparent from the followingdetailed description of the various embodiments of the presentdisclosure with reference to the drawings wherein:

FIG. 1 is a simplified diagram of a system for creating and viewingrenderings of real-world locations with embedded multimedia elements,according to an embodiment of the present disclosure;

FIG. 2 is a simplified box diagram of an exemplary computing deviceforming part of the system of FIG. 1;

FIG. 3 is a flowchart of an exemplary method for creating and viewingrenderings of real-world locations with embedded multimedia elements,according to an embodiment of the present disclosure; and

FIGS. 4A and 4B are example renderings of a real-world location withembedded multimedia elements, according to an embodiment of the presentdisclosure;

FIGS. 5A and 5B are further example renderings of a real-world locationwith embedded multimedia elements, according to an embodiment of thepresent disclosure;

FIGS. 6A and 6B are still further example renderings of a real-worldlocation with embedded multimedia elements, according to an embodimentof the present disclosure;

FIG. 7 is another example rendering of a real-world location withdifferent multimedia elements embedded therein, according to anembodiment of the present disclosure; and

FIGS. 8A and 8B are further example renderings of a real-world locationwith various multimedia elements embedded therein, according to anembodiment of the present disclosure.

DETAILED DESCRIPTION

In the Summary section above, in this Detailed Description, in theClaims below, and in the accompanying drawings, reference is made toparticular features (including method steps or acts) of the presentdisclosure. It is to be understood that the disclosure in thisspecification includes combinations of parts, features, or aspectsdisclosed herein. For example, where a particular feature is disclosedin the context of a particular aspect or embodiment of the presentdisclosure, or a particular claim, that feature can also be used, to theextent possible, in combination with and/or in the context of otherparticular aspects and embodiments of the present disclosure, and in thedisclosure generally.

The term “comprises” and grammatical equivalents thereof are used hereinto mean that other components, ingredients, steps, acts, etc. areoptionally present. For example, an article “comprising (or “whichcomprises”) component A, B, and C can consist of (i.e., contain only)components A, B, and C, or can contain not only components, A, B, and Cbut also one or more additional components, elements, features,ingredients, steps, acts, etc.

Where reference is made herein to a method comprising two or moredefined steps or acts, the defined steps or acts can be carried out inany order or simultaneously (except where the context excludes thatpossibility); and the method can include one or more other steps or actswhich are carried out before any of the defined steps or acts, betweentwo of the defined steps or acts, or after all the defined steps or acts(except where the context excludes that possibility).

When, in this specification, a range is given as “(a first number) to (asecond number)” or “(a first number) (a second number),” this means arange whose lower limit is the first number and whose upper limit is thesecond number. For example, 25 to 100 mm means a range whose lower limitis 25 mm, and whose upper limit is 100 mm.

The present disclosure relates to the visualization of social mediacontent in an immersive, context-aware, and location-aware display.Briefly stated, social media content, such as pictures, drawings,photographs, pictographs, videos, textual messages, comments, audiomessages, etc., (referred to collectively hereinafter as “multimediaelements) may be analyzed to determine their relevance to a particularreal-world location. A rendering may then be created wherein themultimedia elements are embedded into image data of the real worldlocation. A user may view the rendering and thereby experience themultimedia elements in an immersive, context-aware, and location-awaredisplay.

With reference to the drawings, FIG. 1 shows an exemplary system 100 forcreating a rendering of real-world locations with multimedia elementsembedded therein. System 100 may include, among other things, a userdevice 10 including a display, and a computing device 20 connected to aserver 30 via a network 50.

User device 10 may be any of a variety of user devices which may beconfigured to display images. In embodiments, user device 10 may be acomputer 10 a, a television or image projection system 10 b, a tablet 10c, a virtual reality or augmented reality device 10 d, a smartphone 10e, and/or any other devices known to those skilled in the art which maybe configured to display images. Virtual reality or augmented realitydevice 10 d may include one or more desktop monitors, mobile phones,tablets, head-mounted displays and/or tiled displays.

Computing device 20 may be any computer, server, processor, or othermultiples or combinations of the same, which may be configured toreceive image data and multimedia elements, analyze and process suchimage data and multimedia elements, and generate images including therendering of a real-world location with the multimedia elements embeddedtherein. While shown in FIG. 1 and described herein as a separate devicefrom user device 10, it is envisioned that computing device 20 and userdevice 10 may be a single device configured to perform the functions ofboth computing device 20 and user device 10. Alternatively, computingdevice 20 may be a server located remotely from user device 10 and maysend a fully generated rendering to user device 10 to simply bedisplayed by user device 10.

Server 30 may be any server or array of servers used by social mediaplatforms to store multimedia elements. For example, server 30 may be aserver used by a social network, such as FACEBOOK®, TWITTER®,INSTAGRAM®, etc., to store multimedia elements associated with userprofiles on the social network. Similarly, server 30 may be a serverused by media organizations, other multimedia content aggregators, orindividual websites that make the multimedia elements stored on theirservers available for use by the public.

Network 50 may be any network or collection of networks used to connectcomputing device 20 to server 30. In an embodiment, user device 10 mayalso be connected to computing device 20 and/or server 30 via network50. Network 50 may include local area networks (LANs) consisting ofwired and/or wireless networks, wide area networks (WANs), a wirelessmobile network, a BLUETOOTH® network, and/or the internet.

Turning now to FIG. 2, there is shown a simplified block diagram ofcomputing device 20. Computing device 20 may include a memory 21, aprocessor 24, a display 26, a network interface 25, an input device 27,and/or an output module 28. Memory 21 may store an application 23 and/ora database 22. Database 22 may store, among other things, image dataand/or multimedia elements. Application 23 may, when executed byprocessor 24, cause display 26 to present user interface 23 a.

Memory 21 may include any non-transitory computer-readable storage mediafor storing data and/or software that is executable by processor 24 andwhich controls the operation of computing device 20. In an embodiment,memory 21 may include one or more solid-state storage devices such asflash memory chips. Alternatively or in addition to the one or moresolid-state storage devices, memory 21 may include one or more massstorage devices connected to the processor 24 through a mass storagecontroller (not shown) and a communications bus (not shown). Althoughthe description of computer-readable media contained herein refers to asolid-state storage, it should be appreciated by those skilled in theart that computer-readable storage media can be any available media thatcan be accessed by the processor 24. That is, computer readable storagemedia includes non-transitory, volatile and non-volatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer-readable instructions, data structures,program modules or other data. For example, computer-readable storagemedia includes RAM, ROM, EPROM, EEPROM, flash memory or other solidstate memory technology, CD-ROM, DVD, Blu-Ray or other optical storage,magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or any other medium which can be used to storethe desired information and which can be accessed by computing device20.

Processor 24 may be any single or collection of processors. Inembodiments, processor 24 may be a central processing unit (CPU)configured to control computing device 20. In further embodiments,processor 24 may be a dedicated graphical processing unit (GPU)specialized to perform graphics processing. Network interface 25 may beconfigured to connect to a network such as network 50. Input device 27may be any device by means of which a user may interact with computingdevice 20, such as, for example, a mouse, keyboard, foot pedal, touchscreen, and/or voice interface. Output module 28 may include anyconnectivity port or bus, such as, for example, parallel ports, serialports, universal serial busses (USB), or any other similar connectivityport known to those skilled in the art.

Regarding FIG. 3, there is shown a flowchart of an exemplary method forcreating a rendering of real-world locations with embedded multimediaelements. As noted above, user device 10 and computing device 20 mayeach be a distinct device, or may be a single device performing thefunctions of both devices. For purposes of clarity, the examplesdescribed below refer to user device 10 and computing device 20 asdistinct devices. However, these examples are not intended to belimiting, and those skilled in the art will recognize that the same orsimilar functions described below may be performed by a single, unifieddevice.

Starting at step 302, computing device 20 may receive image data of areal-world location. The image data may be received from a server whereit is being stored, and/or may be received directly from a capturingdevice that acquired the image data. In embodiments, the image data ofthe real-world location may be GOOGLE® STREET VIEW® image data and/orMICROSOFT® BING® STREETSIDE® image data. In other embodiments, the imagedata of the real-world location may be acquired by a user, for example,by using user device 10. The image data may include still images such aspanoramic photographs and/or digitally fused or “stitched-together”photographs of a real-world location. The image data may further includeone or more videos of the real-world location. For example, the imagedata may include monoscopic, stereoscopic, and/or multiscopic imagesand/or videos, including 360 degree panoramic videos of a real-worldlocation. The image data may also include live images of a real-worldlocation.

Next, at step 304, computing device 20 identifies geographic coordinatesof the real-world location. The geographic coordinates may be identifiedbased on metadata associated with the image data, for example geo-tags.The geographic coordinates may also be identified by analyzing the imagedata and determining a geographic location based on the image data.

Thereafter, or concurrently with step 304, computing device 20identifies a point of view from which the image data was acquired atstep 306. The point of view may be identified based on metadataassociated with the image data. The point of view may also be identifiedby analyzing the image data and determining the point of view based onthe image data.

Next, at step 308, computing device 20 obtains multimedia elementsrelevant to the real-world location. The multimedia elements may beobtained from server 30, as described above. The relevancy of themultimedia elements to the real-world location may be determined basedon the geographic coordinates identified at step 304 and/or the point ofview identified at step 306. The multimedia elements may be any of thesocial media content described above, such as pictures, drawings,graffiti, gestures, photographs including 2D photos, 3D photos,stereoscopic photos, and/or panoramic photos, videos, computer-generatedvirtual reality or augmented reality renderings, audio recordings, audiomessages, textual messages including TWEETS®, blog posts, instantmessages, text memos, etc., and/or sentiments derived from any of thesemedia types, such as based on text, emoticons, facial expressions, andthe like.

At step 310, computing device 20 filters the obtained multimediaelements based on user-defined parameters. For example, the user mayspecify that only multimedia elements shared by family members, aspecific person or group of people, or a particular public figure shouldbe included, such that a filter parameter may be defined based on theuser's preferences. The filter may also be defined based on a keyword.In another example, the parameters may limit the multimedia elements toelements that were created on a particular date or during a particularseason or other range of dates, at a particular time of day, within aparticular distance from the real-world location, etc. The parametersmay also restrict the source of the multimedia elements, such that, forexample, only multimedia elements posted to FACEBOOK® may be used.Additionally, the parameters may restrict the type of multimediaelements that may be used, for example, only photographs.

Thereafter, or concurrently with the execution of any or all of steps304-310, computing device 20 may, at step 312, determine a geometry ofthe real-world location in the image data. For example, if thereal-world location depicted in the image data is an urban environment,as shown in FIG. 4B, computing device 20 may determine that there arebuildings in the image data and may determine the geometry of suchbuildings. Various image processing algorithms may be used to determinethe geometry of the real-world location.

Next, at step 314, computing device 20 aligns the multimedia elementswith the geometry of the real-world location to create a rendering ofthe image data with the multimedia elements embedded therein. Forexample, computing device 20 may align images with the sides ofbuildings or other objects in the image data. In another example,computing device 20 may create virtual billboards in the image data andalign textual information such as TWEETS® with the virtual billboards.Aligning the multimedia elements with the geometry of the real-worldlocation may be performed using various image processing algorithms,including maximal Poisson sampling among others, and may further bebased on depth and normal maps, road orientations, image saliency, etc.,in a manner that reduces visual clutter.

In further embodiments, the multimedia elements may be presented withrespect to any real-world feature, i.e., aligned to the side of a road,a billboard, a bus-stop, a traffic sign, and/or any other landscapefeature. In other embodiments, the multimedia elements may appear to befloating in mid-air relative to the user's point of view and not alignedto any geometric feature of the landscape.

Thereafter, computing device 20 may send the created rendering of theimage data with the multimedia elements embedded therein to user device10. User device 10 may then, at step 316, display the rendering of theimage data with the multimedia elements embedded therein.

As noted above, user device 10 may be a virtual reality or augmentedreality device 10 d, which may be used to display the rendering of thereal-world location with the multimedia elements embedded therein. Insuch embodiments, a truly immersive, three-dimensional rendering of thereal-world location may be displayed to the user, with the multimediaelements embedded in 360 degrees around the user. Among the variousfeatures which may be used in virtual and/or augmented realityembodiments are the performance of various actions and/or gestures by auser to view and/or interact with the rendering of the real-worldenvironment and/or the multimedia elements embedded therein. Forexample, multimedia elements may be locked or have access restricted,and may be unlocked by a user providing, such as by entering orspeaking, a password, and/or by performing a particular action and/orgesture, as described below with reference to FIG. 8. In such anexample, the user may be required to view a particular real-worldlocation, and perform actions and/or gestures as shown by a multimediaelement to unlock the social media content protected by the gesture.

In a further embodiment, it is envisioned that a user, using user device10, may capture live images of a real-world location where the user islocated, and, by using the captured live images as the image datareceived in step 302 of the above-described method, allow computingdevice 20 to create a rendering of the real-world location using liveimages, and embedding the multimedia elements in such live images.

Turning now to FIGS. 4A and 4B, there is shown example renderings ofreal-world locations with multimedia elements embedded therein. In FIG.4A, image data 410 of an urban setting is shown with multimedia elements415, in this instance photographs that were taken around or inside of abuilding, embedded in the image data so as to appear on the outside ofthe building. Similarly, in FIG. 4B, image data 410 of another urbansetting is shown with multimedia elements 415 embedded in the image dataso as to appear on the outside of the building. Multimedia elements 415are arranged, for example, by using maximal Poisson sampling, such as toreduce visual clutter in the rendering.

In FIGS. 5A and 5B, there is shown further example renderings ofreal-world locations with multimedia elements embedded therein. In FIG.5A, image data 510 of a coastal road is shown with a multimedia element515, in this instance a photograph taken beside the road, embeddedtherein to appear at the geographic location where the photograph wastaken. Likewise, in FIG. 5B, image data 510 of another coastal road isshown with multimedia elements 515, in this instance photographs takenbeside the road and a map of the area, embedded therein and arranged ina manner to reduce visual clutter, as is shown with the map appearing inthe top-right corner of the image data.

The systems, devices, and method of the present disclosure may also beused inside buildings, as is shown in FIGS. 6A and 6B. FIG. 6A shows anexample rendering of a real-world location with multimedia elements 615embedded therein. In this instance, image data 610 of a stage is shownwith multimedia elements 615, in this instance photographs that weretaken on or near the stage, embedded therein and arranged in a manner toreduce visual clutter and not obstruct the focal point of the image,i.e., the middle of the stage. In FIG. 6B, image data 610 of the insideof a stadium is shown with multimedia elements 615, in this instancephotographs that were taken inside the stadium, embedded therein.

FIG. 7 shows another example rendering of an urban environment. Here,image data 710 of the outside of a building is shown with multimediaelements embedded therein. In this instance, multimedia elements such asaudio recordings 720, textual messages 725, videos 730, and graffiti 735are shown as embedded in image data 710.

FIGS. 8A and 8B demonstrate another embodiment of the presentdisclosure. FIG. 8A shows image data 810 of a rural landscape withmultimedia elements 815 shown alongside a trail. A particular multimediaelement may be locked, as described above, with the gesture to beperformed by the user required to unlock the multimedia element shown asgesture 840. After the user performs the required gesture, themultimedia element may be unlocked and displayed as text message 845, asshown in FIG. 8B.

Although the present disclosure has been described in considerabledetail with reference to certain preferred version thereof, otherversions are possible and contemplated. Therefore, the spirit and scopeof the appended claims should not be limited to the description of thepreferred versions contained therein.

Any element in a claim that does not explicitly state “means for”performing a specified function or “step for” performing a specifiedfunction, is not to be interpreted as a “means” or “step” clause asspecified in 35 U.S.C. § 112(f). In particular, the use of “step of” inthe claims is not intended to invoke the provisions of 35 U.S.C. §112(f).

What is claimed is:
 1. A method for creating a rendering of real-worldlocations with embedded multimedia elements, the method comprising:receiving image data of a real-world location; identifying geographiccoordinates of the real-world location and/or a point of view from whichthe image data was acquired; acquiring multimedia elements relevant tothe real-world location based on the geographic coordinates and/or thepoint of view; and creating a rendering of the image data with themultimedia elements embedded therein, wherein the multimedia elementsare embedded in the image data in a manner which fuses the multimediaelements with the image data, wherein fusing the multimedia elementswith the image data includes: determining a geometry of the real-worldlocation based on the image data; and aligning the multimedia elementswith the determined geometry of the real-world location, wherein thealigning includes maximal Poisson sampling.
 2. The method according toclaim 1, further comprising displaying the rendering of the image datawith the multimedia elements embedded therein.
 3. The method accordingto claim 1, wherein the multimedia elements are geo-tagged, and whereinthe multimedia elements are acquired based on geo-tag information of themultimedia elements.
 4. The method according to claim 1, wherein themultimedia elements include pictures.
 5. The method according to claim1, wherein the multimedia elements include videos.
 6. The methodaccording to claim 1, wherein the multimedia elements include textualmessages, gestures, graffiti, or audio messages.
 7. The method accordingto claim 1, wherein the multimedia elements include sentiments derivedfrom pictures, videos, textual messages, gestures, graffiti, or audiomessages.
 8. The method according to claim 1, wherein aligning themultimedia elements with the determined geometry of the real-worldlocation includes displaying the multimedia elements on the sides ofbuildings in the image data.
 9. The method according to claim 1, whereinaligning the multimedia elements with the determined geometry of thereal-world location includes displaying the multimedia elements onvirtual billboards in the image data.
 10. The method according to claim1, wherein the image data include still images of the real-worldlocation.
 11. The method according to claim 1, wherein the image datainclude video images of the real-world location.
 12. The methodaccording to claim 1, wherein displaying the rendering of the image datawith the multimedia elements embedded therein includes displaying therendering using a virtual reality or augmented reality display.
 13. Themethod according to claim 1, wherein the multimedia elements arereceived from social media platforms.
 14. The method according to claim1, further comprising filtering the acquired multimedia elements basedon user-defined parameters.
 15. The method according to claim 14,wherein the user-defined parameters include a date, a time, a season, aperson, group of people, a social media profile, a keyword, a distancefrom the real-world location, or a data type of the multimedia elements.16. The method according to claim 1, wherein the image data includes 360degree panoramic video images of the real-world location.
 17. A systemfor creating a rendering of real-world locations with embeddedmultimedia elements, the system comprising: a computing device includinga processor and a memory storing instructions which, when executed by atleast one processor, causes the computing device to: receive image dataof a real-world location; identify geographic coordinates of thereal-world location and/or a point of view from which the image data wasacquired; acquire multimedia elements relevant to the real-worldlocation based on the geographic coordinates and/or the point of view;create a rendering of the image data with the multimedia elementsembedded therein; and send the rendering of the image data with themultimedia elements embedded therein; and a display device configuredto: receive the rendering of the image data with the multimedia elementsembedded therein; and display the rendering of the image data with themultimedia elements embedded therein, wherein the multimedia elementsare embedded in the image data in a manner which fuses the multimediaelements with the image data, wherein when fusing the multimediaelements with the image data, the instructions, when executed by theprocessor, cause the computer to: determine a geometry of the real-worldlocation based on the image data; and align the multimedia elements withthe determined geometry of the real-world location, wherein the aligningincludes maximal Poisson sampling.
 18. The system according to claim 17,wherein the display device is a virtual reality or augmented realitydisplay device.
 19. A non-transitory computer-readable storage mediumstoring instructions which, when executed by at least one processor,cause a computer to: receive image data of a real-world location;identify geographic coordinates of the real-world location and a pointof view from which the image data was acquired; acquire multimediaelements relevant to the real-world location based on the geographiccoordinates and the point of view; create a rendering of the image datawith the multimedia elements embedded therein; and display the renderingof the image data with the multimedia elements embedded therein, whereinthe multimedia elements are embedded in the image data in a manner whichfuses the multimedia elements with the image data, wherein when fusingthe multimedia elements with the image data, the instructions, whenexecuted by the processor, cause the computer to: determine a geometryof the real-world location based on the image data; and align themultimedia elements with the determined geometry of the real-worldlocation, wherein the aligning includes maximal Poisson sampling.